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VIDEO STORAGE AND RETRIEVAL APPARATUS 
BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an apparatus and 
5 method for retrieving a scene that a user desires, based 
on identification information inserted in digital video 
signals, from a recording media with the digital video 
signals recorded therein, to reproduce. 
Description of the Related Art 

10 Japanese Unexamined Patent Publication 8-130701 

discloses a video retrieving apparatus that searches a 
desired scene based on the contents of the moving video 
to reproduce. FIG.l is a schematic configuration 
diagram of a video retrieving apparatus described in 

15 Japanese Unexamined Patent Publication 8-130701. 

In FIG.l, recorded in storage section 10 is video 
data comprised of bitstreams conforming to MPEG 
(International Standard for Moving Picture Coding). 
Identification information indicative of the contents 

20 of the scene corresponding to the user area is inserted 
in the user data area in the bitstreams. MPEG decoder 
20 has a user area code extraction section 28 that 
extracts the identification information inserted in the 
user data area. Retrieval section 30 is comprised of 

25 retrieval information input section 32 to which the 
contents of the scene a user desires are input, comparison 
section 31 which compares the contents input from 
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retrieval information input section 32 to the extracted 
contents by the user area code extraction section 28. 

When the contents of the desired scene is input from 
retrieval information input section 32, control section 
5 12 is provided with an instruction to retrieve the video 
signal having the identification information matching 
the retrieval scene from video signals in storage section 
11. Then, when the retrieval scene and identification 
information match, retrieval section 30 provides an 

10 instruction indicative of ON to MPEG decoder 20, and then 
the decoding and indication of the corresponding video 
signals (scene) are initiated. 

The case expected as an example of using the video 
retrieving apparatus is that a desired scene is retrieved 

15 from a program while another program is reproduced. For 
example, there is the case that a user searches the 
tomorrow weather forecast from a recorded news program 
to reproduce while reproducing, for example, a recorded 
drama program. Such a case demands a user interface 

20 which executes the retrieval without discontinuing the 
reproduced drama program, and starts reproducing the 
video of the retrieved weather forecast immediately 
after the retrieval is finished or after the reproduced 
program is finished. To achieve such a user interface, 

25 it is necessary to execute processing for the video 
reproduction and retrieval in parallel. 

To execute the retrieval of the video from the 
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identification information inserted in the bitstreams 
conforming to MPEG, it is necessary to extract the data 
from the user data area in the bitstreams. However, it 
is not possible for one MPEG decoder as in the video 
5 retrieving apparatus illustrated in FIG.l to execute the 
processing for the video reproduction and retrieval in 
parallel . 



SUMMARY OF THE INVENTION 

10 The present invention is carried out in view of the 

foregoing, and an object of the present invention is to 
provide a video strage and retrieval apparatus capable 
of executing the reproduction of a video and the retrieval 
of a desired scene in parallel, and providing excellence 

15 as a user interface with good operability provided. 

In various data streams including an MPEG stream, 
compressed coded video streams, audio streams, and 
private streams to store user data are multiplexed 
(hereinafter, the private stream is called retrieval 

20 data stream since identification information for 
retrieval is used as the user data in the present 
invention ) . 

Execution of the reproduction and retrieval from 
the above-mentioned data streams requires a 
25 demultiplexing function for demultiplexing the 
multiplexed data streams to the private streams, video 
streams and audio streams, a video decoding and audio 
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decoding function for extending the compressed coded 
video and audio data to decode, and a retrieval function 
for executing the retrieval. Accordingly, to execute 
the reproduction and retrieval in parallel, it is 
5 necessary to concurrently process streams being 
reproduced and streams for retrieval, however the 
function requiring concurrent processing is only the 
demultiplexing function. 

The present invention enables the concurrent 
.y 10 execution of the reproduction of video and retrieval by 

UJ providing a plurality of demultiplexing sections. 

W 

:J1 BRIEF DESCRIPTION OF THE DRAWINGS 

i;3 The above and other objects and features of the 

: ; 5 

iU 15 invention will appear more fully hereinafter from a 

q consideration of the following description taken in 

connection with the accompanying drawing wherein one 
example is illustrated by way of example, in which; 

FIG.l is a schematic configuration diagram to 
20 explain a conventional video storage and retrieval 
apparatus ; 

FIG. 2 is a schematic configuration diagram of a 
video storage and retrieval apparatus according to a 
first embodiment; 
25 FIG.3A is a diagram illustrating an exemplary 

structure of a MPEG stream according to the above 
embodiment; 
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FIG.3B is a diagram illustrating an exemplary 
structure of video signal; 

FIG.3C is a diagram illustrating an exemplary 
structure of GOP; 
5 FIG. 4 is a schematic configuration diagram of a 

video storage and retrieval apparatus according to a 
second embodiment ; 

FIG. 5 is a schematic configuration diagram of a 
video storage and retrieval apparatus according to a 
10 third embodiment; 

FIG.6A is a diagram illustrating an exemplary 
structure of a MPEG stream according to the third 
embodiment ; 

FIG.6B is a diagram illustrating an exemplary 
15 structure of a demultiplexed data stream; 

FIG.6C is a diagram illustrating an exemplary 
structure of a data stream for retrieval; 

FIG. 7 is a schematic configuration diagram of a 
video storage and retrieval apparatus according to a 
20 fourth embodiment; 

FIG. 8 is a schematic configuration diagram of a 
video storage and retrieval apparatus according to a 
fifth embodiment; 

FIG. 9 is a schematic configuration diagram of a 
25 video storage and retrieval apparatus according to a 
sixth embodiment; 

FIG. 10 is a schematic configuration diagram of a 
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video storage and retrieval apparatus according to a 
seventh embodiment ; 

FIG. 11 is a schematic configuration diagram of a 
video storage and retrieval apparatus according to an 
5 eighth embodiment; and 

FIG. 12 is a schematic configuration diagram of a 
video storage and retrieval system according to a ninth 
embodiment . 

10 DETAILED DESCRIPTION OF THE 

PREFERRED EMBODIMENTS 
Embodiments of the present invention will be 
described below with reference to accompanying drawings . 
The following description explains as an example of data 

15 stream the MPEG bitstream conforming to the MPEG that 
is the moving picture coding standard. However, it is 
obvious that the present invention is applicable to the 
data stream other than the MPEG stream conforming to the 
MPEG that is the moving picture coding standard. 

20 (First embodiment) 

FIG. 2 is a block diagram of a video storage and 
retrieval apparatus according to the first embodiment 
of the present invention. The video storage and 
retrieval apparatus is comprised of MPEG stream input 

25 section 1 to which the MPEG stream is input, video output 
section 2 which outputs a decoded video signal, and audio 
output section 3 which outputs a decoded audio signal. 
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The MPEG stream received in MPEG stream input section 
1 is stored in storage section 11. Storage section 11 
has a plurality of output ports, and the recorded 
plurality of MPEG streams can be accessed concurrently. 
5 Storage section 11 is controlled by control section 12. 
Control section 12 controls the read of the MPEG stream 
stored in storage section 11 according to the instruction 
from retrieval section 30. 

Two output ports of storage section 11 are coupled 
.Tj 10 to respective first demultiplexing section 21 and second 

iy demultiplexing section 22 which can be accessed 

\j concurrently. First demultiplexing section 21 

■ - 2 

iff demultiplexes video streams and audio streams from the 

q MPEG stream read from storage section 11 to reproduce, 

m 15 Second demultiplexing section 22 demultiplexes 

i;n retrieval streams form the MPEG stream read from storage 

sect ion 1 1 to retrieve . The video streams , audio streams 
and retrieval streams are multiplexed in the MPEG stream. 

First demultiplexing section 21 is connected to 
20 video decoding section 23 and audio decoding section 24 
in parallel. Video decoding section 23 extends the 
compressed video streams to decode, and outputs the 
decoded signal to video output section 2 . Audio decoding 
section 24 extends the compressed audio streams to decode, 
25 and outputs the decoded signal to audio output section 
3. Video decoding section 23 and audio decoding section 
24 operate synchronously. 
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Retrieval section 30 is comprised of comparison 
section 31 and retrieval information input section 32. 
Retrieval information input section 32 receives the 
inputted retrieval information) contents of the scene). 
5 Comparison section 31 compares the identification 
information in the retrieval data stream, which is read 
from storage section 11 and then demultiplexed in second 
demultiplexing section 22 , to the retrieval scene from 
retrieval information input section 32, while 
:.fj 10 instructing control section 12 on the read of the MPEG 

hj stream to retrieve. 

i : y 

y FIG.3A illustrates an exemplary structure of the 

Ly 

m MPEG stream in this embodiment. As illustrated in 

□ FIG.3A, the MPEG stream is comprised of stream header 
| = y 15 101 to store stream information, a plurality of coded 

□ video streams (video stream (1) 102 to video stream (n) 
105), a plurality of coded audio streams (audio stream 
(1) 103 to audio stream (n) 106), and a plurality of 
retrieval data streams (retrieval data stream (1) 104 

20 to retrieval data stream (n) 106) as identification data 
with identification information for retrieval 
incorporated therein. 

The identification information indicative of the 
contents of the corresponding scene is recorded in the 
25 retrieval data stream along with PTS (Presentation Time 
Stamp) that is information to synchronously reproduce 
the corresponding video data streams and audio data 
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streams, or time code. And each of the video data 
streams, the audio data streams and the retrieval data 
streams is constructed in packets and multiplexed as PES 
(Packetized Elementary Stream), Further, it may be 
5 possible to insert the retrieval data streams as a 
private descriptor in the PAT (Program Association 
Table), PMT (Program Map Table) or CAT (Conditional 
Access Table) to multiplex with the video data and audio 
data . 

i.p 10 FIG.3B is an exemplary structure of the video 

Id signals comprised of a plurality of GOPs (Group of 

vj Picture) comprised of coded video data, and FIG.3C 

i a n illustrates an exemplary structure of a GOP header. The 

q picture sequence illustrated in FIG.3B is constructed 

ry 15 in packets, and stored as element data in video stream 

□ (1) 102 to video stream (n) 105 in the MPEG stream 

illustrated in FIG.3A. 

The GOP is a group comprised of a series of I picture, 
P picture and B picture obtained by coding timewise 
20 successive picture frames (or fields). In an apparatus 
for coding a video signal to record and reproduce, the 
GOP is comprised of the I picture obtained by coding image 
in a frame (or field) , the P picture obtained by executing 
motion compensation inter-frame (or picture) coding 
25 referring to the I picture, another P picture obtained 
by executing motion picture compensation inter-frame (or 
picture) referring to the P picture obtained by executing 
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the inter-frame (or picture) coding as described above, 
and the B picture obtained by executing bidirectional 
motion compensation inter-frame (or field) coding 
referring to I pictures or P pictures between which the 
5 corresponding picture (or frame) exists timewise 
successively . 

As illustrated in FIG.3B, the sequence of coded 
video signals is comprised of a sequence header 201 and 
a plurality of GOPs (first GOP="GOPl", thereafter, the 
!.?} 10 data number is assigned to the same type of data in the 

l e y same way as described above) 202-1 to 202-n. FIG.3C 

J illustrates a detail of the inside of GOP2 ( 202-2) . G0P2 

Iff is comprised of header 203, one or a plurality of I 

O pictures 206 , a plurality of P pictures 209 , 212 and 215, 

ry 15 and a plurality of B pictures 207, 208 , 210 , 211 , 213, 

Q 214, 216, 217 and 218. The numbers described in 

; c 

respective parenthesis each indicates a head address of 
each data. 

The processing is executed on a GOP-by-GOP basis 
20 in decoding the video signal. Video decoding section 23 
executes decoding of pictures input in the order of II, 
B-l, B0, P4, B2, B3 • • • . The decoded video signal is 
output to a display device through video output section 
2, and displayed in the order of frames (B-l), (B0), II, 

25 B2 , B3 , P4 • • * . 

In addition, the PTS to synchronously reproduce the 
video signal and audio signal is inserted to respective 
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header areas of video stream (1) 102 to video stream (n) 
105 and audio stream (1) 103 to audio stream (n) 106. 
The time stamps is inserted in a header area (203) of 
G0P1 (202-1) to GOPn (202-n). 
5 The operation of the apparatus of this embodiment 

with the configuration as described above is next 
explained. When control section 12 receives a 

reproduction instruction from a user interface, not 
shown in the figure, with a program address or program 

10 identification number designated, control section 12 
provides the reproduction instruction to storage section 
11 while specifying a header address of the MPEG stream 
of the instructed program. As a result of the 
instruction, the MPEG stream of the program instructed 

15 by control section 12 is provided from one of output ports 
of storage section 11 to first demultiplexing section 
21. First demultiplexing section 21 demultiplexes the 
video streams and audio streams from the MPEG stream. 
The demultiplexed video streams are decoded in video 

20 decoding section 23, and the demultiplexed audio streams 
are decoded in audio decoding section 24. Then, the 
video signal and audio signal are synchronously output 
from video output section and audio output section 3. 

Thus, the present invention enables the retrieval 

25 of the desired data stream from storage section 11 even 
when the program is reproduced. In other words, when the 
contents of a scene to be retrieved is input to retrieval 
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information input section 32 of retrieval section 30 from 
a user, comparison section 31 instructs control section 
12 so that the MPEG stream for retrieval is output from 
storage section 11. At this time, second demultiplexing 
5 section 22 accesses another output port other than the 
output port outputting the MPEG stream being reproduced. 
Second demultiplexing section 22 demultiplexes the 
retrieval data stream from the MPEG stream for retrieval. 
Then, the identification information inserted in the 
10 retrieval data stream is output from second 
Ljj demultiplexing section 22 to comparison section 31. 

y Comparison section 31 compares the retrieval scene 

i : f| input to retrieval information input section 32 to the 

•3 

O identification information in the retrieval data stream, 

ry 15 and instructs control section 12 to read out the MPEG 

q stream stored in storage section 11 until the retrieval 

scene and the identification information match. 

Further, second demultiplexing section 22 refers 
to the PTS or a value of the time code stored in the 
20 retrieval data stream, and calculates the GOP head 
address of the corresponding picture frames, and the 
number of the corresponding picture frames starting from 
the GOP head frame, to output to retrieval section 30. 

When the retrieval scene and identification 
25 information match in comparison section 31, retrieval 
section 30 holds the GOP head address of the corresponding 
picture frames and the number of the corresponding 
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picture frames starting from GOP head frame. Then, 
retrieval section 30 notifies control section 12 of the 
GOP head address and the number of the frames from the 
GOP head frame when instructs control section 12 to 
5 reproduce the retrieved scene. Thereby, it is possible 
to access directly the desired picture frame when the 
reproduction of the retrieved picture starts. 

As described above, according to this embodiment, 
since first demultiplexing section 21 and second 

10 demultiplexing section 22 are provided, it is possible 
to concurrently execute the reproduction and retrieval. 

In addition, it may be possible to provide a 
plurality of second demultiplexing sections 22 that 
execute the demultiplexing to extract the retrieval 

15 information and a plurality of retrieval sections 30 that 
search specific video and audio data based on the 
identification data so that the plurality of second 
demultiplexing sections 22 and retrieval sections 30 
execute the retrieval processing in parallel on a 

20 plurality of data streams stored in storage section 11. 
Thereby, it is possible to speed up the retrieval 
processing . 

Further, it may be possible that the data stream 
is input to second demultiplexing section 22, which 
25 executes the demultiplexing to extract the retrieval 
information, at a rate higher than the data transfer rate 
at which the data stream is input to first demultiplexing 



# • 

14 



section 21 which executes the demultiplexing to 
reproduce the video and audio, in order to speed up the 
retrieval processing, 
( Second embodiment ) 
5 FIG, 4 is a block diagram of a video storage and 

retrieval apparatus according to the second embodiment 
of the present invention. In addition, the section with 
the same function as in FIG. 2 is given the same mark as 
in FIG, 2 to omit the specific explanation thereof in the 

5 10 following. 

i n 

Ly In the video storage and retrieval apparatus of this 

•J embodiment, third demultiplexing section 25 is provided 

\M between MPEG stream input section 1 and storage section 

O 11. Third demultiplexing section 25 demultiplexes an 

ry 15 input multiplexed MPEG stream to video streams, audio 

q streams and retrieval data streams. The demultiplexed 

video streams, audio streams and retrieval data streams 
are recorded in storage section 11. Storage section 11 
has three output ports. The output ports of storage 
20 section 11 are respectively connected to video decoding 
section 23, audio decoding section 24 and comparison 
section 31 of retrieval section 30. 

The operation of the video storage and retrieval 
apparatus according to the second embodiment is 
25 explained below. The multiplexed MPEG stream is input 
to MPEG stream input section 1 , and demultiplexed in third 
demultiplexing section 25 to video streams, audio 
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streams and retrieval data streams, which are stored in 
storage section 11. 

At the time the video is reproduced, the video 
streams are input to video decoding section 23, while 
5 the audio streams are input to audio decoding section 
24 , both from storage section 11, and the video and audio 
are synchronously output from respective sections. 

In addition, at the time the video is retrieved, 
the retrieval data streams stored in storage section 11 

10 are input to retrieval section 30, and therein the same 
retrieval processing as in the first embodiment is 
executed. Comparison section 31 searches the retrieval 
data stream in which the identification information 
matches the retrieval scene, while referring to the PTS 

15 or a value of the time code stored in the retrieval data 
stream, and calculating the GOP head address of the 
corresponding picture frames, and the number of the 
corresponding picture frames from the GOP head frame. 
The calculated values are used in reproducing the 

20 retrieved scene. 



streams and retrieval data streams demultiplexed from 
the MPEG stream are stored in storage section 11, while 
the video streams, audio streams and searches data stream 
25 stored in storage section 11 are made independently 
accessible, and therefore it is possible to execute the 
reproduction of the video and audio, and the retrieval 



As described above. 



the video streams, 



audio 
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at the same time. 
( Third embodiment ) 

FIG. 5 is a block diagram of a video storage and 
retrieval apparatus according to the third embodiment 
5 of the present invention. In addition, the section with 
the same function as in FIG. 2 is given the same mark as 
in FIG. 2 to omit the specific explanation thereof in the 
following. In the video storage and retrieval apparatus 
of this embodiment, fourth demultiplexing section 26 is 

10 provided between MPEG stream input section 1 and storage 
section 11, and fifth demultiplexing section 27 is 
provided between storage section 11, and video decoding 
section 23 and audio decoding section 24. Fourth 
demultiplexing section 26 demultiplexes the multiplexed 

15 MPEG stream to a data stream in which video streams and 
audio streams are multiplexed, and retrieval data 
streams. Fifth demultiplexing section 27 demultiplexes 
the multiplexed video streams and audio streams to video 
streams and audio streams. 

20 FIG.6A illustrates a structure of the MPEG stream 

input to fourth multiplexing section 26, and FIG.6B 
illustrates structures of the MPEG streams output from 
fourth demultiplexing section 26. 

The operation of the video storage and retrieval 

25 apparatus according to this embodiment is explained 
below. The multiplexed MPEG stream, input to MPEG stream 
input section 1, is demultiplexed to video streams, audio 
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streams and retrieval data streams in fourth 
demultiplexing section 26, which multiplexes again 
demultiplexed video streams and audio streams, while 
adding stream header 111 thereto, to output. The 
5 retrieval data streams and multiplexed video streams and 
audio streams are stored in storage section 11. 

At the time the video is reproduced, the multiplexed 
video streams and audio streams, stored in storage 
section 11, are demultiplexed to the video streams and 
;.n 10 audio streams by fifth demultiplexing section 27. The 

I rs 

!,sj video streams and audio streams are converted into video 

ru 

y signals and audio signals in video decoding section 23 

UT and audio decoding section 24, and output therefrom, 

□ respectively. 

W ■ 

fH 15 At the time the video is retrieved, the retrieval 

p data streams stored in storage section 11 are input to 

: sssF 

retrieval section 30, and the retrieval processing is 
executed therein. At this time, the retrieval data 
streams, and the multiplexed video streams and audio 

20 streams can be accessed independently, and accordingly, 
it is possible to execute the reproduction of the video 
and audio, and the retrieval at the same time. 
( Fourth embodiment ) 

FIG. 7 is a block diagram of a video storage and 

25 retrieval apparatus according to the fourth embodiment 
of the present invention. In addition, the section with 
the same function as in FIG. 2 is given the same mark as 
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in FIG. 2 to omit the specific explanation thereof in the 
following . 

In the video storage and retrieval apparatus of this 
embodiment, temporary storage section 40 is provided 
5 between storage section 11, and first demultiplexing 
section 21 and second demultiplexing section 22, At the 
time the reproduction or retrieval is executed, the MPEG 
date stored in storage section 11 is transferred to 
temporary storage section 40 at a data transfer rate 

10 higher than that necessary for the ordinary video 
reproducing processing to be temporarily stored therein, 
and the stored data is provided to first demultiplexing 
section 21 and second demultiplexing section 22 at the 
same data transfer rate as in the ordinary reproducing 

15 processing. 

The operation of the video storage and retrieval 
apparatus in this embodiment is explained below. At the 
time of video reproduction, part of or entire MPEG stream 
to be reproduced is transferred to temporary storage 

20 section 40 from storage section 11 at the data transfer 
rate higher than that necessary for the ordinary video 
reproducing processing. Then, the data is transferred 
to first demultiplexing section 21 at the same data 
transfer rate as in the ordinary reproducing processing 

25 from temporary storage section 40. First demultiplexing 
section 21 demultiplexes video streams and audio streams 
to respectively output to video decoding section 23 and 
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audio decoding section 24 . The decoded video signals and 
audio signals are respectively output from video output 
section 2 and audio output section 3. 

Meanwhile, control section 12 controls storage 
5 section 11 so that storage section 11 transfers again 
the rest of the MPEG stream to be reproduced to temporary 
storage section 40 before the MPEG stream to be reproduced 
stored in temporary storage section 40 is all transferred 
to first multiplexing section 21. 

10 At the time of video retrieval, a user inputs the 

retrieval request to retrieval section 30. In this case, 
control section 12 judges whether the MPEG stream to be 
reproduced is being transferred from storage section 11 
to temporary storage section 40, and when the MPEG stream 

15 is not being transferred, the MPEG stream for the 
retrieval is transferred from storage section 11 to 
temporary storage section 40. Second demultiplexing 
section 22 fetches the retrieval data streams from the 
MPEG stream for the retrieval stored in temporary storage 

20 section 40, and outputs the identification information 
in the retrieval data stream to retrieval section 30. 

While the data stream for the video reproduction 
is being transferred to temporary storage section 40 from 
storage section 11, it is not possible to read out the 

25 data stream for the retrieval from storage section 11. 
However, after all the data steams for the video 
reproduction are transferred to temporary section 40, 
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or at periods between which the transfer of the data 
streams for the video reproduction is executed 
periodically, it is possible to read out the data stream 
for the retrieval from storage section 11 even while the 
5 video is being reproduced. 

In the casd where storage section 11 is, for example , 
a medium that can be accessed only sequentially such as 
VTR, it is not possible to concurrently access a plurality 
of MPEG streams stored in storage section 11 even with 

;.n 10 first demultiplexing section 21 and second 

Id demultiplexing section 22 provided. 

ru 

y Therefore, as described above, in the case where 

!jf storage section 11 is the medium that can be accessed 

O only sequentially, by providing a recording medium 40 

ry 15 enabling concurrent accesses to a plurality of MPEG 

Q streams next to an output of storage section 11, it is 

possible to access the plurality of MPEG streams at the 
same time, and therefore to execute the reproduction of 
video and audio, and the retrieval at the same time. 
20 Further, since the MPEG stream is transferred to 

temporary storage section 40 from storage section 11 at 
the data transfer rate higher than that for the ordinary 
video reproducing processing, the MPEG stream 
temporarily stored in temporary storage section 40 is 
25 not fully consumed, thus enabling the continuous 
reproduction. 
(Fifth embodiment ) 
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FIG. 8 is a block diagram of a video storage and 
retrieval apparatus according to the fifth embodiment 
of the present invention. In addition, the section with 
the same function as in FIG. 2, FIG. 4 or FIG. 7 is given 
the same mark as in FIG. 2, FIG. 4 or FIG. 7 to omit the 
specific explanation thereof in the following. 

In the video storage and retrieval apparatus of this 
embodiment, the MPEG stream, in which video streams, 
audio streams and retrieval data streams are multiplexed, 
is read from storage section 11, input to third 
demultiplexing section 25 to be demultiplexed to the 
video streams, audio streams and retrieval data streams. 
Then, the video streams and audio streams are stored in 
temporary storage 40, and at the time of retrieval, the 
retrieval data streams are input to retrieval section 
30. The temporary storage section 40 outputs the video 
streams and audio streams in parallel respectively to 
video decoding section 23 and audio decoding section 24. 

The operation of the video storage and retrieval 
apparatus of this embodiment is explained below. At the 
time of video reproduction, part of or entire MPEG stream 
to be reproduced is transferred to third demultiplexing 
section 25 from storage section 11 at the data transfer 
rate higher than that for the ordinary video reproducing 
processing. Third demultiplexing section 25 

demultiplexes the multiplexed video streams, audio 
streams and retrieval data streams to the video streams 
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and audio streams, and the retrieval data streams, and 
transfers the video streams and audio streams to 
temporary storage section 40 to temporarily store 
therein. Then, temporary storage section 40 transfers 
5 the video streams and audio streams respectively to video 
decoding section 23 and audio decoding section 24 at the 
same data transfer rate as in the ordinary reproducing 
processing. Video decoding section 23 and audio 
decoding section 24 respectively output the decoded 
uq 10 video signals and audio signals. 

j s y Meanwhile, control section 12 controls storage 

rU 

Sj section 11 so that storage section 11 transfers again 

Iff the rest of the MPEG stream to be reproduced to temporary 

G storage section 40 via third demultiplexing section 25 

;U 15 before the video streams and audio streams to be 

O reproduced stored in temporary storage section 40 are 

all transferred respectively to video decoding section 
23 and audio decoding section 24. 

When a user inputs the retrieval request to 
20 retrieval section 30, control section 12 judges whether 
the MPEG stream to be reproduced is being transferred 
from storage section 11 to temporary storage section 40, 
and when the MPEG stream is not being transferred, the 
MPEG stream for the retrieval is transferred from storage 
25 section 11 to third demultiplexing section 25. 

Third demultiplexing section 25 demultiplexes the 
retrieval data streams from the MPEG stream for the 
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retrieval, and transfers the retrieval data stream to 
retrieval section 30 to be subjected to retrieve 
processing . 

As described above, in this embodiment, since the 
retrieval data streams are not stored in temporary 
storage section 40, it is possible to use temporary 
storage section 40 of which the capacity is less than 
that of temporary storage section 40 in the fourth 
embodiment . 

In addition, in the above-mentioned embodiment, it 
may be possible that temporary storage section 40 is 
comprised of a random access memory device, the 
identification data demultiplexed from the MPEG stream 
as an object for the retrieval is also stored in temporary 
storage section 40, and that retrieval section 30 
executes the retrieval processing using the 
identification data read from temporary storage section 
40 . 

Further, in the above-mentioned embodiment, it may 
be possible that temporary storage section 40 is 
comprised of a random access memory device, third 
demultiplexing section 25 demultiplexes the MPEG stream 
as an object to be reproduced and/or for retrieval to 
the identification data, and streams of multiplexed 
video data and audio data, temporary storage section 40 
stores the streams of video data and audio data, and the 
decoding section comprised of video decoding section 23 
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and audio decoding section 24 decodes the streams of video 
data and audio data* 

Furthermore, in the above-mentioned embodiment, it 
may be possible that temporary storage section 40 is 
5 comprised of a random access memory device, third 
demultiplexing section 25 demultiplexes the MPEG stream 
as an object to be reproduced and/or for retrieval to 
the identification data, and streams of multiplexed 

video data and audio data to respectively store in 

a 

kQ 10 temporary storage section 40. Then, the decoding 

(n 

Ly section comprised of video decoding section 23 and audio 

%i decoding section 24 decodes the streams of video data 

J , 5 

iff and audio data, while retrieval section 30 executes the 

□ retrieval processing using the identification data read 

Ly 

ry 15 from temporary storage section 40. 

ru 

( Sixth embodiment ) 

FIG. 9 is a block diagram of a video storage and 
retrieval apparatus according to the sixth embodiment 
of the present invention. In addition, the section with 
20 the same function as in FIG. 2 or FIG. 7 is given the same 
mark as in FIG. 2 or FIG. 7 to omit the specific explanation 
thereof in the following. 

The video storage and retrieval apparatus of this 
embodiment has temporary storage section 40 provided 
25 between storage section 11 and first demultiplexing 
section 21 in the configuration of the first embodiment. 

The operation of the video storage and retrieval 
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apparatus in this embodiment is explained below. At the 
time of video reproduction, the MPEG stream stored in 
storage section 11 is transferred to temporary storage 
section 40 from storage section 11 at the data transfer 
5 rate higher than that for the ordinary video reproducing 
processing. Then, the MPEG stream is transferred to 
first demultiplexing section 21 from temporary storage 
section 40 at the same data transfer rate as in the 
ordinary reproducing processing. First demultiplexing 

O 

i.g 10 section 21 demultiplexes video streams and audio streams 

LR 

Ly from the MPEG stream, and outputs video streams and audio 

y streams respectively to video decoding section 23 and 

i.n audio decoding section 24. Thereby, the decoded video 

O signals and audio signals are output. 

fy 15 Meanwhile, control section 12 controls storage 

p section 11 so that storage section 11 transfers again 

the rest of the MPEG stream to be reproduced to temporary 
storage section 40 before the MPEG stream to be reproduced 
stored in temporary storage section 40 is all transferred 
20 to first multiplexing section 21. 

When a user inputs the retrieval request to 
retrieval section 30, control section 12 judges whether 
the MPEG stream to be reproduced is being transferred 
from storage section 11 to temporary storage section 40, 
25 and when the MPEG stream is not being transferred, the 
MPEG stream for the retrieval is transferred from storage 
section 11 to second multiplexing section 22. Second 
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demultiplexing section 22 demultiplexes the retrieval 
data streams (or identification information) from the 
MPEG stream to output to retrieval section 30 . Retrieval 
section 30 executes the retrieval processing using the 
5 identification information and retrieval scene. 

With respect to the data stream comprised of 
multiplexed video streams, audio streams and retrieval 
data streams, when the data amount of the retrieval data 
streams is smaller than that of the video streams and 

Q 

; g 10 audio streams, second demultiplexing section 22 can 

Ly execute the demultiplexing processing for 

ro 

demultiplexing the retrieval data streams sufficiently 

! - £ 

Lfi faster than the processing time required for the video 

□ reproduction, 

i ! y 15 Accordingly, even if storage section 11 is a medium 

O that can be accessed only sequentially such as a VTR, 

it is possible to access a plurality of MPEG streams at 
the same time. 
( Seventh embodiment ) 
20 FIG. 10 is a block diagram of a video storage and 

retrieval apparatus according to the seventh embodiment 
of the present invention. In addition, the section with 
the same function as in FIG. 2 or FIG. 4 is given the same 
mark as in FIG. 2 or FIG. 4 to omit the specific explanation 
25 thereof in the following. 

The video storage and retrieval apparatus of this 
embodiment has second storage section 13 provided 
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between third demultiplexing section 25 and retrieval 
section 30 in the configuration of the second embodiment. 
Second storage section 13 is comprised of a recording 
device that can be accessed randomly such as a 
5 semiconductor memory, HDD, DVD-RAM, PD and optical- 
magnetic disk alone or in thereof. 

The operation of the video storage and retrieval 
apparatus according to the above embodiment is explained 
below. The multiplexed MPEG stream input to MPEG stream 

3 10 input section 1 is demultiplexed in third demultiplexing 

in 

Ly section 25 to video streams, audio streams and retrieval 

%f data streams. The demultiplexed video streams and audio 

Lf! streams are stored in storage section 11, and the 

0 demultiplexed retrieval data streams are stored in 

1 ti 

| 5 y 15 second storage section 13 . 

I y . 

O At the time of video reproduction, the video streams 

and audio streams, stored in storage section 11, are 
respectively input to video decoding section 23 and audio 
decoding section 24, which respectively output the video 

20 and audio. On the other hand, at the time of retrieval, 
the retrieval data streams, stored in second storage 
section 13, are input to retrieval section 30, which 
executes the retrieval processing. 

As described above, the multiplexed MPEG stream is 

25 demultiplexed to the video streams, audio streams and 
retrieval data streams before being stored in storage 
sections, and the video streams and audio streams used 
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for the video reproduction are stored in storage section 
11, and the retrieval data streams used for the retrieval 
are stored in another storage section, i.e., second 
storage section 13, thereby it is possible to 
5 concurrently execute the video reproduction and 
retrieval in parallel even if storage section 11 is the 
medium that can be accessed only sequentially such as 
a VTR . Further, since only the retrieval data streams 
are stored in second storage section 13 comprised of the 

10 recording device that can be accessed randomly, it is 
possible to execute fast video retrieval. 

In addition, in the above-mentioned embodiment, it 
may be possible to provide fourth demultiplexing section 
26 illustrated in FIG. 5 before storage section 11 and 

15 second storage section 13 instead of third 
demultiplexing section 25, and further provide fifth 
demultiplexing section 27 illustrated in FIG. 5 between 
storage section 11, and video decoding section 23 and 
audio decoding section 24. Then, retrieval section 30 

20 may execute the retrieval processing while fetching the 
identification data from second storage section 13. 
According to this configuration, it is also possible to 
execute the fast retrieval, and to concurrently execute 
the video reproduction and retrieval in parallel, even 

25 if storage section 11 is comprised of the medium that 
can be accessed only sequentially such as the VTR. 

Further, in the above-mentioned embodiment, it may 
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be possible that retrieval section 30 executes the 
retrieval processing while fetching the identification 
data from second storage section 13, selects a period 
matching the pre - regis tered retrieval condition, and 
5 stores the streams of video data and audio data 
demultiplexed from the MPEG stream corresponding to the 
selected period in storage section 11. 

Furthermore, in the above-mentioned embodiment, it 
may be possible to provide fourth demultiplexing section 
yg 10 26 illustrated in FIG. 5 before storage section 11 and 

Uj second storage section 13 instead of third 

-J demultiplexing section 25, and further provide fifth 

\M demultiplexing section 27 illustrated in FIG. 5 between 

O storage section 11, and video decoding section 23 and 

|y 15 audio decoding section 24. Then, retrieval section 30 

□ may execute the retrieval processing while fetching the 

identification data from second storage section 13, 
select a period matching the pre-registered retrieval 
condition registered, and store the streams of video data 
20 and audio data demultiplexed from the MPEG stream 
corresponding to the selected period in storage section 
11. Otherwise, retrieval section 30 may execute the 
retrieval processing while fetching the identification 
data from fourth demultiplexing section 26 not through 
25 second storage section 13, select a period matching the 
per- regis tered retrieval condition, and store the video 
streams and audio streams demultiplexed from the MPEG 
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stream corresponding to the selected period in storage 
section 1 1 . 
(Eighth embodiment ) 

FIG. 11 is a block diagram of a video storage and 
5 retrieval apparatus according to the eighth embodiment 
of the present invention. In addition, the section with 
the same function as in FIG, 2 or FIG. 4 is given the same 
mark as in FIG. 2 or FIG. 4 to omit the specific explanation 
thereof in the following. 

10 The video storage and retrieval apparatus of this 

embodiment has the configuration of the second 
embodiment, in which the retrieval data streams output 
from third demultiplexing section 25 are directly input 
to retrieval section 30. 

15 The operation of the video storage and retrieval 

apparatus according to this embodiment is explained 
below. The multiplexed MPEG stream input to MPEG stream 
input section 1 is demultiplexed in third demultiplexing 
section 25 to the video streams, audio streams and 

20 retrieval data streams. The demultiplexed retrieval 
data streams are input to directly retrieval section 30 
not through storage section 11. 

In this embodiment, the retrieval condition input 
from retrieval information input section 32 is 

25 registered in advance in retrieval section 30. 
Retrieval section 30 compares the identification 
information contained in the retrieval data stream input 
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from third demultiplexing section 25 to the pre- 
registered retrieval condition. Then, when the 

information and condition match, retrieval section 30 
outputs a matching signal to control section 12. 

When the matching signal is output from retrieval 
section 30, control section 12 controls storage section 
11 to store the video streams and audio streams 
demultiplexed from the corresponding MPEG stream in 
storage section 11. As a result, the video and audio at 
the period matching the condition desired by a user are 
only stored in storage section 11. 

As described above, the retrieval data streams are 
demultiplexed before the MPEG stream is stored in storage 
section 11, and input to retrieval section 30, and the 
period corresponding to the retrieval data stream 
matching the pre-registered retrieval condition is only 
stored in storage section 11, thereby enabling the 
efficient video recording. 

In addition, in the above-mentioned embodiment, it 
may be possible to provide fourth demultiplexing section 
26 illustrated in FIG. 5 before storage section 11 instead 
of third demultiplexing section 25, and further provide 
fifth demultiplexing section 27 illustrated in FIG. 5 
between storage section 11, and video decoding section 
23 and audio decoding section 24. Then, retrieval 
section 30 may execute the retrieval processing while 
fetching the identification data from fourth 
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demultiplexing section 26, select a period matching the 
pre - regis tered retrieval condition, and store the 
streams of video data and audio data demultiplexed from 
the MPEG stream corresponding to the selected period in 
5 storage section 11. 
(Ninth embodiment ) 

FIG. 12 is a block diagram of a video storage and 
retrieval system according to the ninth embodiment. In 
addition, the section with the same function as in FIG. 2 
10 or FIG. 5 is given the same mark as in FIG. 2 or FIG. 5 to 
omit the specific explanation thereof in the following. 

The video storage and retrieval system of this 
embodiment is comprised of video retrieval server 50, 

15 video reproducing terminal 51 and transmission media 52. 

Video retrieval server 50 is comprised of MPEG 
stream input section 1, fourth demultiplexing section 
25, storage section 11, control section 12, comparison 
section 31, MPEG stream transmission section 4 which 

20 transmits the MPEG stream comprised of multiplexed video 
data streams and audio data streams, and retrieval 
information reception section 6 which receives the 
retrieval information (scene contents). 

Video reproducing terminal 51 is comprised of fifth 

25 demultiplexing section 27, video decoding section 23, 
audio decoding section 24, retrieval information input 
section 32, video output section 2, audio output section 
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3, MPEG stream reception section 5 which receives the 
MPEG stream comprised of multiplexed video data streams 
and audio data streams, and retrieval information 
transmission section 7 which transmits the retrieval 
information . 

Further, transmission media 52 between video 
retrieval server 50 and video reproducing terminal 51 
are comprised of communication networks, broadcasting 
networks, or a combination of the communication networks 
and broadcasting networks. 

The operation of the video storage and retrieval 
system in this embodiment is explained below. When a 
user inputs the contents to be retrieved from retrieval 
information input section 32 in video reproducing 
terminal 51, the input retrieval information is 
transmitted from retrieval information transmission 
section 7 through transmission media 52 to video 
retrieval server 50. 

In video retrieval server 50, retrieval information 
reception section 6 receives the retrieval information. 
The received retrieval information is input to 
comparison section 31. Comparison section 31 compares 
the identification information in the retrieval data 
streams stored in storage section 11 to the received 
retrieval information to execute the video retrieval. 
The video matching the retrieval information is 
transmitted as multiplexed video and audio streams from 
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MPEG stream transmission section 4 through transmission 
media 52 to video reproducing terminal 51. 

In video reproducing terminal 51, MPEG stream 
reception section 5 receives the retrieved video and 
5 audio streams. The received video and audio streams are 
demultiplexed in fifth demultiplexing section 27, and 
the demultiplexed video streams and audio streams are 
respectively expanded in video decoding section 23 and 
audio decoding section 24, thus outputting the videos 
i.g 10 and audioes. 

UJ Further, since the retrieval data streams are 

%J demultiplexed and stored in storage section 11, it is 

in possible to concurrently execute the processing fast for 

O the retrieval requests from a plurality of terminals. 

f 5 j 

ry 15 In addition, when at least the storage section and 

ru 

O retrieval section, and at least the video decoding 

section and audio decoding section in the video storage 
and retrieval apparatus described in one of the first 
to eighth embodiments and various modifications thereof 

20 are respectively provided at the video retrieval server 
side, and at the video reproducing terminal, it may be 
possible to provide the other components at either side, 
enabling the appropriate design and/or modification. 

As described above, according to the present 

25 invention, it is possible to concurrently execute the 
video reproducing processing and retrieval processing 
in parallel . 



35 



Further, according to the present invention, even 
in the case where the medium that can be accessed only 
sequentially such as a VTR is used as the storage section, 
it is possible to concurrently execute the video 
5 reproducing processing and retrieval processing. 

The present invention is not limited to the above 
described embodiments, and various variations and 
modifications may be possible without departing from the 
scope of the present invention, 
10 This application is based on the Japanese Patent 

Application No . HE 1 1 1 - 8 4 3 3 7 filed on March 26, 1999, 
entire contents of which is expressly incorporated by 
reference herein. 



